Sharpened Error Bounds for Random Sampling Based $\ell_2$ Regression

Abstract. Given a data matrix X ∈ R and a response vector y ∈ R , suppose n > d, it costs O(nd) time and O(nd) space to solve the least squares regression (LSR) problem. When n and d are both large, exactly solving the LSR problem is very expensive. When n ≫ d, one feasible approach to accelerating LSR is to randomly embed y and all columns of X into the subspace R where c ≪ n; the induced LSR problem has the same number of columns but much fewer number of rows, and the induced problem can be solved in O(cd) time and O(cd) space. The leverage scores based sampling is an effective subspace embedding method and can be applied to accelerate LSR. It was shown previously that c = O(dǫ log d) is sufficient for achieving 1 + ǫ accuracy. In this paper we sharpen this error bound, showing that c = O(d log d + dǫ) is enough for 1 + ǫ accuracy.